1. Field of the Invention
The invention relates to a process for transforming and routing data between agent servers present in some machines and a central agent server present in another machine.
More specifically, the invention relates to a process for transforming and routing data between agent servers present in machines of production sites and a central agent server present in the machine of the decision-making base, for example within the framework of a centralized concentration of data, also known as a Data Warehouse.
2. Description of Related Art
Data Warehousing is a computing concept aimed at concentrating in a centralized unit, accessible through a network, all of the information available in a company. Data Warehousing allows a better distribution of the information by no longer presenting the information in the form of raw data, but so that it is directly usable by the requestor or client (the company""s executives, for example). A Data Warehouse therefore corresponds to one or more of the company""s production sites with databases, plus a decision-making base that can retrieve the information in a new form (matching information that can be represented in different ways, transforming or deleting information, etc.). At each site, there is a process for transforming and routing the data for filling this decision-making base. The hardware and software used can be different from one site to another. In the prior art, the machines at each site require a particular software program for interpreting and transforming the information from the database or databases of the site. A software program must therefore be rewritten for each site. The processes that currently exist use the distribution to the various machines of a synchronous process known as a Remote Procedure Call (RPC). The various elementary operations (the extraction, transfer and loading stages) are mixed in one algorithm.
The object of the invention is to restructure the data transforming and routing software in order to make it easily extendable and to make it open.
This object is achieved through the fact that the process for transforming and routing data between agent servers present in some machines and a central agent server present in another machine is characterized in that an agent server comprises autonomous agents that communicate via notifications, a storage layer (Storage) and an agent machine comprising an execution engine (Engine), a communication channel (Channel) and two message queues for the notifications, a local queue (mqIn) and an external queue (mqOut), and in that the execution engine takes a notification from the local queue (mqIn), determines and loads the corresponding agent for each notification, has the agent react to the notification, which agent can then change its status and/or send notifications to the communication channel, which stores them in the local queue (MqIn) if they are addressed to a local agent and in the external queue (MqOut) if they are addressed to an agent of another server.
According to one characteristic, the execution engine (Engine) of the agent machine executes, after the agent""s reaction, a save (Save) of the agent""s status, a deletion (Pop) of the processed notification from the local message queue (MqIn) and, atomically, a validation (Commit), of the various operations performed that have changed either the status of the agent or the status of the message queues.
According to another characteristic, each message includes an optimized matrix clock that can be read by the execution engine of the agent machine, thus making it possible to process the notifications in their causal sending order.
According to another characteristic, a message is an object that contains four pieces of information, i.e., the identifier of the sending agent, the identifier of the destination agent, a clock and the notification.
According to another characteristic, the agent machine controls the distribution to all the entities of the network, the reliability of the operations and the scheduling of the messages, which is established by incrementing a logical matrix clock.
According to another characteristic, the message channel performs the following separate transactions:
a notification sent by an agent is stored locally in a message queue, i.e., either the local queue (mqIn) for the notifications addressed to a local agent, or the external queue (mqOut) for an agent located in another server;
at the end of the agent""s reaction, the two queues are saved on the disk, together with the reacting agent;
a function (handleout) of the network extracts each notification, in turn, from the external queue (mqOut) and sends it to the target agent server;
a function (handlein) of the network receives the notifications from the other servers and inserts them into the local queue (mqIn) in causal order.
According to another characteristic, a target agent server receives a message, stores it in its persistent storage layer (Storage), validates the operation (Commit) and confirms the reception of the message to the source server; upon reception of this confirmation, the source server deletes the message from its external queue (mqOut).
According to another characteristic, a persistent external service ensures the atomic nature of the operations that have changed either the status of the agent or the status of the message queues (3,4).
According to another characteristic, the execution engine:
takes a notification that is in the queue, using the Get function,
sees which agent the notification is addressed to,
loads the agent corresponding to the notification (Load),
has the agent react by calling a reaction operation (React); the agent can change its status by modifying its image in memory, and can send notifications by addressing the channel that will distribute the notification, in which case the notifications are routed to a message queue where they are stored in chronological order,
if the reaction operation is executed, saves (Save) the status of the agent, which has been changed by the reaction it has executed, deletes (Pop) the notification that has been processed, and atomically validates all of the operations that have been performed (Commit),
starts again in the program loop in order to process another notification.
According to another characteristic, the data transforming and routing process is executed between agent servers present in machines of production sites and a central server present in the machine of a decision-making base, and uses a particular agent (Event) describing the specific events, in which a status automation is implemented.
According to another characteristic, the status controller comprises various statuses:
a None status in which the event is not active,
a notification (Schedule) causes the event to pass into the Wait status in which the event waits for its activation condition to become true; the event delivers itself a notification that changes it to the Init status;
a simple event attains the Run status directly and ail executes its action; a composed event signals to its sub-events its agreement for them to start and waits for the first of them to attain the Run status in order to pass into the Run status itself;
a simple event then passes into the Done or Fail status, depending on the status of its action; for a composed event, this change depends on the execution statuses of all the sub-events;
a event in execution can be killed by sending it a notification (Eventkill), in which case it passes into the Kill status, marking the fact that it has been killed, and then to the Stop status when it has actually been killed;
a terminated event, i.e., in the Done, Fail or Stop status, can be re-executed by the client by sending it a notification (Schedule), in which case it is reset and passes into the Wait status; the event then propagates the notification to all the sub-events; an event executed repeatedly and endowed with a Done status is automatically reset by the agent Scheduler, which sends it a notification (Schedule);
an event in the Fail or Stop status can be re-executed without being reset by sending a notification (Eventrestart); when this condition becomes xe2x80x9ctrue,xe2x80x9d the event restarts at the point at which it was previously stopped.
According to another characteristic, a new function can be added to the architecture by adding an agent class derived from the class xe2x80x9cEvent.xe2x80x9d
According to another characteristic, three categories of xe2x80x9cEventxe2x80x9d type agentsxe2x80x94xe2x80x9cSeqEvent,xe2x80x9d xe2x80x9cParEvent,xe2x80x9d xe2x80x9cComplexEventxe2x80x9dxe2x80x94type take on other elementary operations of the xe2x80x9cEventxe2x80x9d type in order to respectively execute them in sequence, in parallel or in accordance with the dependencies between the various operations.
By Other characteristics and advantages of the present invention will emerge more clearly through the reading of the following description provided in reference to the appended drawings, in which:
FIG. 1 represents the architecture of an agent server at a site and the algorithm of the execution engine of the agent machine;
FIG. 2 represents the status controller of an agent xe2x80x9cEventxe2x80x9d;
FIGS. 3, 4 and 5 respectively represent the class xe2x80x9cAgent,xe2x80x9dthe class xe2x80x9cChannel,xe2x80x9d and the class xe2x80x9cmessageQueue.xe2x80x9d